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ABSTRACT 


We  consider  an  M/M/1  queue  which  is  controlled  by  dynamically 
setting  the  customer's  entrance  fee  to  either  or  b2>bi  anc* 
thereby  setting  the  customer  arrival  rate  to  either  or  X2<x^ , 
respectively.  With  E [ F ] the  expected  fee  collected  per  unit  of 
time  and  the  steady  state  probability  that  the  system  contains 
more  than  N customers,  we  consider  two  criteria:  (i)  for  some 
number  r,  minimize  subject  to  E[F]>r,  and  (ii)  for  some  number 
e,  maximize  E[F]  subject  to  P^<e.  Under  each  criterion  we 
consider  two  cases:  (a)  the  admissible  policies  are  single 
critical  number  switching  policies  and  (b)  a cost  y>0  is 
incurred  whenever  the  server  switches  between  b^  and  b£,  the 
admissible  policies  allow  for  hysteresis  to  appear  in  the 
arrival  process.  Optimal  policies  are  computed  for  each  criterion 
and  each  case. 


nI  One  can  observe,  in  the  literature  on  queueing  theory,  a 
growing  interest  in  the  control  of  the  arrival  process.  For 


most  of  the  published  models,  the  aim  is  to  maximize  the 
difference  between  fees  collected  and  costs:  it  is  explicitly 
assumed  that  the  fees  and  costs  can  be  measured  and  are  expressed 
in  the  same  units.'  They  are  then  merged  into  a single  objective 
function  (Naor  [7],  Yechiali  [12,13],  Edelson  & Hildebrand  [1], 
Teghem  [10]  or  Low  [5,6],  among  others.) 

In  fact,  it  is  recognized  that  such  a merging  cannot  always 
be  done.  Just  to  give  one  example,  we  observe  that  the  fee 
charged  to  customers  may  be  an  arbitrary  notion  (as  noted  by 
Nielsen  [8]:  "the  pricing  of  computer  services  is  not  dependent 
upon  charging  users  real  money");  in  such  a case,  it  might  be 
difficult  to  express  costs  in  the  same  unit  as  the  fee.  We 
assume  in  this  paper  that  the  reason  for  controlling  the  arrival 
process  is  to  achieve  some  balance  between  acceptance  and 
rejection  of  customers;  while  letting  customers  in  increases 
the  fee  collected,  an  excess  of  customers  should  be  avoided. 

WeNconsiderA-'the  two  objectives  of  maximizing  the  fee  collected 
and  minimizing  queue  congestion  as  distinct.  If  it  is  not 
possible  to  optimize  these  simultaneously,  then  the  problem  will 
be  to  maintain  one  of  them  at  a reasonable  level  and,  under  that 
constraint,  to  optimize  the  other.  ^ 

In  another  paper  [4],  we  considered  similar  models  where  the 
two  objectives  are  merged  into  one  single  function.  The  analysis 
and  results  in  [4]  are  both  quantitatively  and  qualitatively 
different  from  those  we  present  here. 
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Specifically,  we  assume  that  the  server  may  dynamically 
set  the  entrance  fee  to  either  or  bg>b^  ; the  arrivals  form  a 
Poisson  process  with  parameter  \ 1 or  x2<x.  respectively. 

The  service  times  are  Independent,  identically  distributed, 
exponential  (y)  random  variables,  independent  of  the  arrival 
process . 

The  waiting  room  is  infinite,  but  the  total  number  of 
customers  in  the  system  should  preferably  not  exceed  some  value 
N,  called  the  critical  level. 

With  E[F],  the  expected  fee  collected  per  unit  of  time,  in 
steady  state  and  PN  the  steady  state  probability  that  the  system 
contains  more  than  N customers,  we  consider  two  criteria: 

(i)  for  some  number  r minimize  P^ , subject  to  E[F]>r, 
and 

(ii)  for  some  number  e maximize  E[F],  subject  to  PN<e. 

Under  each  criterion,  we  consider  two  cases: 

(a)  the  admissible  policies  are  single  critical  number 
switching  policies,  they  are  characterized  by  a number  M:  if 
there  are  fewer  than  M customers  in  the  system,  the  entrance  fee 
is  b.j , otherwise  it  is  b2; 

(b)  a cost  y>0  is  incurred  whenever  the  server  switches 
between  b-j  and  b 2 and  the  admissible  policies  allow  for  hysteresis 
to  appear  in  the  arrival  process.  These  policies  are  characterized 
by  two  numbers,  m and  M (msM-1):  if  the  fee  is  b-j  and  the 

number  of  customers  increases  from  (M-l)  to  M,  the  fee  becomes  b2; 
if  the  fee  is  b2  and  the  number  of  customers  decreases  from  (m+1 ) 
to  m,  the  fee  becomes  b-j. 


A 


Hysteresis  in  the  service  process  has  been  considered  by 
several  authors  (for  instance,  Yadin  & Naor  [11]  or  Teghem  [10]). 
To  the  best  of  our  knowledge,  only  Scott  [9]  has  considered 
hysteresis  in  the  arrival  process;  we  comment  more  on  this  paper 
in  Section  4. 

Optimal  policies  are  computed  for  each  criterion  and  each 

case. 

The  policies  we  determine  are  not  necessarily  optimal  with 
respect  to  larger  sets  of  admissible  control  policies.  For 
instance,  we  mention  in  Sections  6 and  7 that  even  with  y=0, 
some  policy  with  hysteresis  may  be  strictly  superior  to  every 
single  critical  number  policy.  Moreover,  one  can  determine 
randomized  stationary  policies  which  are  strictly  superior  to 
any  policy  in  the  classes  we  have  considered.  Nevertheless, 
these  policies  are  easily  Implemented  and  can  be  efficient,  as 
we  show  by  examples  in  Section  8. 

The  optimal  single  critical  number  policy  is  determined 
explicitly  for  the  first  criterion;  for  the  second  criterion,  the 
optimal  policy  Is  determined  explicitly  for  large  e,  through  an 
equation  with  a unique  solution  for  small  e (this  is  made  precise 
in  Section  3).  To  find  the  optimal  policy  with  hysteresis,  we 
use  properties  of  E[F]  and  to  determine  a finite  number  of 
pairs  (m,M)  among  which  the  best  is  to  be  found  numerically. 

The  computations  involved  are  straightforward.  We  shall  not 
present  detailed  proofs;  some  of  them  are  quite  lengthy  and  are 
presented  in  [3]. 
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To  conclude  the  introduction,  we  comment  on  our  measure  of 
queue  congestion,  the  probability  of  having  more  than  N 
customers  in  the  system.  In  some  cases,  a more  appropriate 
measure  might  be  the  expected  queue  length  or  the  mean  cost  of 
waiting  if  customers  were  able  to  measure  such  a cost. 

For  the  first  criterion,  the  optimal  single  critical  number 
policy  would  be  the  same;  it  appears  in  Section  2 that  the 
optimal  value  for  M does  not  depend  on  N.  In  the  other  cases, 
we  suspect  that  the  analysis  would  follow  the  same  lines  although 
the  details  are  of  course  different. 


5 


1.  Control  With  a Single  Critical  Number 


In  this  case,  the  model  is  simple  to  analyze  and  most  of  the 
results  in  Sections  1 to  3 are  intuitively  obvious.  As  stated  in 
the  introduction,  we  assume  b i < b 2 and  X-^Xg. 

xi 

Moreover,  if  p..*— , i = l,2,  we  assume  that  p2<1  , while  p-|  may 
be  less  than  or  greater  than  1. 

It  is  obvious,  therefore,  that  steady  state  conditions  are 
satisfied  if  M<°°  or  M=»  and  p,<l. 


tt ^ ( M ) = P[i  customers  in  the  system  in  steady  state|M] 


P,(M)  * Z tt.(M)  , 

1 j = i + 1 1 

we  shall  refer  to  the  P. 's  as  the  tail  probabilities. 

One  can  easily  solve  the  system  of  equations  for  the 
probabilities  ^.(M)  (1*0,1,...)  and,  from  there,  one  gets  for  p^l 


Pi(M)  = 


i+1 / , \ M , , 

p-|  (1“P2/"P]  ( p i "P  2 ' 


1-p2-p1  ( p -j  -p 2 ) 


i <M  , 


\ M i-M+1 
(i-p1  ; p 1 p 2 

1 ~ p 2 — p i ( p -|  -p 2 ^ 


i >M ; 


for  p-|  = l,  the  corresponding  formulas  are 

l+(M-i-l)(l-P  ) 

Pi(M)  = 1+M(1-p2) isM’ 

i-M+1 

p2 
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Lemma  1 : For  all  i,  P..(M)  is  an  increasing  function  of  M. 

This  property  is  intuitively  obvious:  if  M increases,  the 
number  of  states  for  which  the  arrival  rate  is  (>X2)  increases 
and  the  probability  of  having  more  than  i customers  increases. 
Now,  let 

r(M)  = E[fee  collected  per  unit  of  time  in  steady  state|M]. 

Of  course, 

r ( M ) = X2b2  + (X1b1-X2b2)(l-PM_1(M)).  (2) 


where 


P, (1-0,  ) 


1 ~p2 "p 1 ^ p 1 ” p 2 ^ 


Lemma  2:  r(M)  is  an  increasing  function  of  M if  x1b^>x2b2;  it  is 
constant  if  x^b^=x2b2  and  decreasing  if  x^b-<x2b2. 

This  results  from  the  fact  that  PM_-|(M)  is  decreasing  in  M 
for  all  finite  value  of  p^.  The  following  result  is  an  immediate 
consequence  of  Lemmas  1 and  2: 


Theorem  1 : If  Xjbj<x2b2,  it  is  optimal  to  set  M equal  to  0. 

In  fact,  only  if  X^b^<X2b2  can  one  reach  simultaneously  the 
two  objectives:  maximize  r(M)  and  minimize  PN(M)  and  it  is  done 
by  choosing  the  fee  b2  always.  We  shall  assume  in  the  next  two 
sections  that  x^b^>x2b2. 


2.  Constraint  on  the  Expected  Fee 

Suppose  it  is  decided  that  the  expected  fee  r(M)  must  be  at 
least  equal  to  some  predetermined  value  r.  The  problem  is  then 


to  find  such  that 


and 


r (Mi )>r 

Pn(M-j  )^Pn(M)  for  all  M such  that  r(M)*r. 

Let  r(«)=lim  r(M),  one  gets  easily  from  (1)  and  (2)  that 


(3) 


lim  PM  ,(M)  = 0 


PT] 

p 1 ”p2 


if  p 1 < 1 » 


» 


hence 


r(»)  = A,b 


n 


p-i  <i  * 


= A„b 


p 1 ~ 1 


+ A,  b. 


1-P. 


2U  r,  » •>  u 1 < A 1 b 1 Pl>l 

2 p -j  - p 2 1 1 p,-p,  1 1 H1 


1 w2 


Clearly,  if  M is  equal  to  the  expected  fee  over  any 
interval  of  time  is  A^b^  times  the  length  of  that  interval,  even 
if  p i > 1 . Therefore,  if  r(M=«)  denotes  the  value  of  r(M)  when  M=<», 
we  shall  consider  that  r(M=~)  = A^b1  for  all  A ^ . 

Theorem  2: 

If  A,b,<r,  then  there  is  no  solution; 


M i =~ ; 


A^b^<r,  then 
r(«)<r<A1b1 , 

A2b2<r<r(»)  , A 2 > 0 Mj-rxjl; 

Osr<r  («>)  , A2  = 0 


r< A2b2 


M-j  =max ( N , Tx-j  1 ) ; 
M -j  = 0 ; 
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where 

x1  = -log(1+y)/logp1  if  p^l, 

= (r-x2b2)/((x1b1-r)(l-p2))  if  p^l  , 

y=((r-x2b2)(1-p1))/((x1b1-r)(1-p2)) 

and  r x 1 denotes  the  smallest  integer  greater  than  or  equal  to  x. 

The  proof  of  this  theorem  is  easy  and  can  be  found  in  [3]. 
Observe  that  if  X2  = 0,  the  level  M takes  on  a special  meaning: 
it  is  the  maximum  number  of  customers  the  server  allows  in  the 
system.  If  x-j  as  defined  in  the  theorem  is  less  than  (N-l),  then 
the  solution  is  not  unique,  since  PN(M)=0  and  r(M)*r  for  all 
M=  f x i ] , Tx1 1+1  ....  ,N.  However,  since  the  critical  level  has 

been  set  to  N and  not  a smaller  value,  should  be  N since 

r(N)>r(N-l  )>. . . Therefore,  if  x2  = 0,  then  =max (N  , Tx^ 1 ) . 

If  X2^0,  then  M.|  does  not  depend  on  N,  therefore,  if  the 
objective  is  to  keep  the  expected  fee  at  the  level  r at  least, 
it  is  not  necessary  to  determine  a critical  level  N: 
minimizes  the  tail  probabilities  P.(M)  for  all  i>0  and  minimizes 
the  expected  queue  length.  To  analyze  as  a function  of  the 
different  parameters,  one  analyzes  either  the  explicit  expression 
for  x1  or  r(M).  It  appears  that  M-j  is,  trivially,  a (non 
strictly)  increasing  function  of  r and  a decreasing  function  of 
all  the  other  parameters:  X^  , x2,  y,  b^  and  b2* 

One  can  show  also  that  if  r is  a linear  combination  of  x^b-| 
and  X2b2 , 

r = wX^b-|  + (l-w)x2b2 

for  some  we(0,l),  then  M-|  is  finite  if  and  only  if  y is  greater 
than  the  same  linear  combination  of  x^  and  x2: 


y>wX-|  + (l-w)x2. 
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Constraint  on  the  Tail  Probabilit 

Suppose  now,  contrary  to  the  preceding  section,  that 
it  is  decided  that  the  probability  of  having  more  than  N customers 
in  the  system  has  to  be  less  than  some  given  value  e.  The  problem 
is  then  to  find  M2  such  that 
Pn(M2)<£ 

and 

r(M2)>r(M)  for  all  M such  that  P N ( M ) < e . 

M2  is  determined  very  easily.  Two  cases  must  be  distinguished: 
if  PN ( N ) < e , it  results  from  Lemma  1 that  M2  is  greater  than  or 
equal  to  N and  is  determined  using  equation  (la);  if  Pn(N)>e,  M2<N 
and  equation  (lb)  has  to  be  used. 

Note  that 

P N ( °° ) = 1 i m pN(M)  = nn'n(l  , p.,N  + 1) 

M-*oo 

and  that  P^(0)=p2  should  be  smaller  than  or  equal,  to  e in 
order  to  have  a solution. 

Theorem  3 

N + l 

If  e<P2  , there  is  no  solution; 

P2  <e<mi n ( 1 ,p1  ) , m2  = Lx2 J ; 

mi n ( 1 ,p^  + 1 ) <e<l  , M2=»; 

where  x2<N  is  the  unique  nonnegative  root  of  the  equation 
(Pl/P2)X=eO-P2-pi(p-|-p2))/(p2  + 1(1_pi)) 
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if  PN ( N ) <e<mi n ( 1 »P^+1  ) (7) 

and  L x J denotes  the  greatest  integer  less  than  or  equal  to  x. 

Proof  of  this  theorem  can  be  found  In  [3]. 

In  the  strict  sense,  equation  (5)  may  have  two  non  negative 
N + 1 

roots  if  e=  p 2 : x=0  and  some  positive  root,  say  x*.  It  can  be 

seen  easily  that  in  this  case,  x*<l  and  therefore  there  is  no 
ambiguity  on  the  value  of  Mg:  both  L 0 J and  |.x*J  are  equal  to  0. 

Observe  that  Xg  does  not  depend  on  b1  or  bg.  Therefore,  if 
the  objective  is  to  keep  the  probability  P N ( M ) at  the  level  e 
at  most  ( e>Pg+^ ) , there  is  no  need  to  determine  precise  values 
for  b1  and  bgj  it  suffices  to  check  whether  X^b^>Xgbg  (it  is 
more  advantageous  to  let  many  customers  in  the  system)  or  not,  in 
the  first  case,  theorem  3 applies,  in  the  second,  theorem  1 and 
Mg=0 . There  is  no  explicit  form  for  Mg  if  Mg<N . However,  it 
is  very  easy  to  determine  numerically  a suitable  approximation 
for  Xg  using  standard  methods;  as  Mg  is  Integer  valued,  one  does 
not  need  a very  high  precision  on  this  approximation  and  it  is 
found  rapidly. 

By  analyzing  P N ( M ) as  a function  of  the  different  parameters, 
one  shows  that,  not  surprisingly.  Mg  is  a decreasing  function  of 
X-|  and  Xg  and  an  increasing  function  of  u,  N and  e. 
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We  shall  examine  now  the  second  type  of  control,  where 
hysteresis  is  introduced  in  the  arrival  process. 


Let  us  assume  x^b^>X2b2*  If  X^b^^X2b2»  it  can  be  shown  that 
the  optimal  rule  is  to  impose  the  fee  b^  always. 

Let  us  assume  furthermore  that  y>0  where  y is  the  cost 
incurred  by  the  server  each  time  there  is  a change  of  fee  and 

M-l*m*0  (8) 

This  last  inequality  means  that  the  fee  is  certainly  b-j  when  the 
system  is  empty  and  the  only  purpose  of  this  assumption  is  to  get 
a more  uniform  presentation  of  the  results. 

The  determination  of  the  state  probabilities  and  the  expected 
reward  is  not  as  straightforward  as  for  the  preceding  type  of 
control.  M.  Scott  [9]  derives  the  probabilities  when  the  maximum 
queue  size  is  finite. 

In  [3],  we  determine  these  quantities  - when  the  maximum 
queue  size  is  infinite  - by  using  a different  technique  that  can 
be  roughly  described  as  follows: 

Let  r(m,M)=E[fee  collected  per  unit  of  time  in  steady  state|(m,M)] 

and  Pi (m,M)=P[more  than  i customers  in  the  system  in  steady 
state| (m,M)]. 

Obviously,  r(m,M)  is  equal  to  the  expected  total  fee  collected 
during  a busy  cycle  divided  by  the  expected  length  of  a busy 
cycle  and  similarly  for  P^(m,M). 

To  study  a busy  cycle,  we  decompose  it  into  homogeneous 


intervals  of  time:  first,  there  is  an  idle  period.  When  a first 
customer  joins  the  queue,  there  is  a random  walk  with  absorbing 
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boundaries  at  0 and  M,  during  this  random  walk,  the  arrival  rate 
is  constant  and  equal  to  , the  fee  is  b-j  . If  aL'orptlon  is  in 

O,  the  busy  cycle  terminates;  if  absorption  is  in  M,  the  server 
suffers  a loss  y and  a new  random  walk  is  initiated,  with 
absorbing  boundary  in  m,  arrival  rate  X2  anc*  ^ee  ^2*  uPon 
absorption  in  m,  there  is  a cost  y,  a new  random  walk  of  the 
first  type  is  initiated  and  the  process  repeats  itself. 

The  detailed  computation  for  r(m,M)  is  given  in  the  appendix; 

P. (m,M)  is  determined  in  much  the  same  way,  as  can  be  found  in  [3]. 

Eventually,  one  gets  for  P.(m,M): 


0<i  <m 


P • ( m , M ) = l M 

(l-pg)  (1-Pi  ) + (M-m)pi ( p 2 ” p i ) ( 1 - p 1 ) 


m- 1 <i <M 


P^m.M): 


(1-p2)2(1-p^  1)+  (M-i)  p^  1 1 ( p 2 - p 1 ) ( 1 -p  1 ) ( 1 -p2  )+P^  1 1 (p2-P2~fn+1  JC^-Py  )‘ 


(1-P2)  (l~P^~m)  + (M-m)p^,(p2-p^  ) (1-p^ ) 


M- 1 < i 


Pi  (m»M)s— 


M i-M+1 

p 1 P 2 


2 (l-P2)0"Pi  m)  + (M-m)p^  (pg-p-j  ) (1 -p-j  ) 


plJj'’1(l-P1)2(2y(l-p2)  + (M-m)(b-|p1-b2P2)) 

r (m,M)=X, b,  - X m m y 

1 ' 1 (1-p2)(1-p,  l + tM-mlplJfpj-p^O-p,) 


and 


r 
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5.  Analysis  of  the  Expected  Fee  and  the  Tall  Probabilities 

Lemma  3:  For  all  i,  P^(m,M)  is  an  increasing  function  of  m and  M. 

This  property  can  easily  be  justified  in  the  same  way  as 
Lemma  1.  Note  that  there  is  no  general  relationship  between 
P.j(m,M)  and  P^(m',M')  if  m<m’  and  M>M’. 

The  behavior  of  r(m,M)  as  a function  of  m and  M is  more 

complex.  Details  can  be  found  in  [3]  but  we  shall  summarize  here 

the  most  important  properties. 

Lemma  4:  For  any  given  m,  r(m,M)  is  an  increasing  function  of  M. 

Lemma  5:  For  any  given  M,  r(m,M)  is  a unimodal  function  of  m and 

we  denote  by  m^  the  corresponding  maximizing  value. 

In  fact,  we  determine  quantities  y^(M)  and  y2(M)  such  that 
if  y<y-|(M)  (the  cost  y is  small)  then  mM=M-l  : for  that  value  of 
M the  reward  is  maximum  when  there  is  no  hysteresis;  if  y>y2(M) 
then  mM=0:  m should  be  as  far  apart  from  M as  possible;  if  y 
lies  between  y^(M)  and  y2(M),  mM  is  determined  by  finding  the 
unique  root  of  some  equation. 

Remark : for  M>1  . 

These  properties  and  others  give  us  detailed  although  not 
complete  information  on  r(m,M). 
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6.  Constraint  on  the  Expected  Fee 

The  problem  is  to  find  (m^,M^)  such  that 

' * 

r(m1  » M -j  )>r 

and  )<PN(m,M)  for  all  (m,M)  such  that  r(m,M)>r. 

Let  Q be  the  set  { (m  ,M ) | M=1  ,2  , . . . ; m=0 , 1 , . . . ,M-1  } of  all 
pairs  satisfying  (8)  and  let 

«1  3 {(m,M) | (m,M)eO,  r(m,M)>r}. 

Obviously,  (m^  ,M1 . 

If  n.|  is  empty,  the  problem  has  no  solution;  if  it 

results  from  Lemma  3 that  (m^  ,M^ ) = (0,1 ) , the  fee  should  be  b2 
whenever  there  is  a customer  in  the  system;  otherwise,  we 
proceed  in  two  steps  to  determine  (m^,M^). 

First,  by  using  repeatedly  the  properties  of  r(m,M),  we 
prove  that  has  the  form  indicated  on  figure  1:  the  dashed 
region  corresponds  to  ; the  boundary  of  is  made  up  of 
two  parts:  for  increasing  m,  it  is  first  non-increasing  and 
second  it  is  non-decreasing  until  it  coincides  with  the  line  m=M-l. 
We  have  marked  by  an  x the  "corners"  in  the  first  part  of  the 
boundary.  We  denote  those  pairs  by  (m*,M*).  We  shall  not  give 
here  the  full  definition  of  the  (m*,M*)’s,  as  it  is  very 
cumbersome.  Let  us  mention  that  they  are  ordered  in  such  a way 
that  and  m*>m*+j  . Their  most  important  properties  are: 

M*=mi n{M j r(mM ,M)>r}  ; 

y 

for  a gi ven  m*m ^ , 

r(m*,M)>r  iff  M*M*  ; 
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for  a given  M=MX, 

y — y 

r(m,M.)<r  for  all  m<m.  . 

We  then  use  Lemma  3 to  prove  the  following  theorem: 

Theorem  4 

(m1,M1)£{(m^,H^),  i=l,...,Ix}. 

There  is  no  explicit  expression  for  one  should 

compare  all  (mx,Mx)'s  to  determine  It.  This  can  easily  be  done 
numerically.  Note  that  Ix  is  at  most  equal  to  Tx^l  (defined  in 
Theorem  2)  and  is,  in  general,  smal 1 er ; we  have  thus  an  upper 
bound  on  the  number  of  pairs  that  will  have  to  be  determined  and 
compared . 

To  analyze  the  solution  as  a function  of  the  parameters,  one 

has  to  resolve  to  computations.  As  an  example,  we  have  examined 

how  (mj»M-|)  depends  on  N (see  figure  2).  For  the  single 

critical  number  policies,  we  have  seen  that  is  independent  of 

N.  For  control  with  hysteresis,  it  appears  that  (m^  ,M^ ) varies 

in  general  only  slightly  with  N,  if  at  all;  however,  if  r is 

very  close  to  the  lim  r(m,M)  and  A.  is  much  larger  than  A,,  the 

M-*»  1 * 

variations  are  greater,  as  can  be  seen  in  figure  2.  Those  cases 

do  not  seem  typical . 

One  can  observe  that  if  y=0,  m^  is  not  necessarily  equal  to 
( M i - 1 ) (Table  I).  In  other  words,  even  if  the  server  suffers  no 
cost  each  time  the  fee  is  changed,  it  is  optimal  in  some  cases 
to  introduce  hysteresis  in  the  arrival  process. 


L J 
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7.  Constraint  on  the  Tail  Probability 
(m2,M2)  is  defined  as  follows 

P ^ (m2  »^2 

and  r(m2,M2)£r(m,M)  for  all  (m,M)  such  that  PN(m,M)<e. 

Let  n2={ (m,M) | (m,M)en  and  PN(m,M)<e}.  If  «2  is  empty,  the 
problem  has  no  solution.  If  ft2=n,  it  results  from  Lemma  4 that 
M2=“  and  therefore  m2  can  take  any  value:  the  fee  is  b-|  always. 
Otherwise,  we  use  Lemma  3 to  prove  that  n2  has  the  form  indicated 
on  figure  3:  the  dashed  region  corresponds  to  the  boundary 

of  fi2  forms  a non-increasing  line  for  increasing  m.  We  have 
marked  by  a + some  of  the  points  in  n2,  denoted  by  (iti|,mT). 

Again,  we  do  not  define  formally  those  pairs  in  this  paper.  Let 
us  mention  that  they  are  ordered  in  such  a way  that  and 

mT>mj+.|.  (m^,M^)  is  defined  as  follows: 

m|  = max{M|PN(mM,M)<e>, 
ml  = mM|  ; 

the  other  pairs  represent  the  "corners"  in  the  boundary  of  n2,  to 
the  left  of  (m|,M|);  moreover,  for  a given  m=m^, 

PN(mt,M)<e  iff  M<Mt 

and  for  a given  M=m|, 

P^ (m  ,mT ) < e iff  m<mt. 

We  then  use  the  properties  of  r(m,M)  to  prove  the  following 


theorem : 
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Theorem  5: 

(m2,M2)e{ (m|,M|) , i=l,...,I+}  . 

To  complete  theorem  5,  we  may  add  that 

if  (mj,M^+l )e«2  , then  (m2 ,M2 )? (mj  ,m|  ) . 

In  other  words,  (m2,M2)  lies  on  the  boundary  of  fl2  and  (mj.Mj) 
cannot  be  optimal  if  it  lies  inside  «2.  x2  (defined  in  Theorem  3) 
provides  an  upper  bound  on  the  number  of  pairs  that  have  to  be 
determined  and  compared,  for  I+<  ( L*2-l  + 1 ) • 

We  have  analyzed  numerically  (m2,M2)  as  a function  of  y and 
it  appears  (figures  4 to  6)  that  M2  is  a (non  strictly)  increasing 
function  of  y while  m2  is  decreasing.  This  property  is  not 
surprising:  if  the  cost  y increases,  the  server  should  wait 

longer  before  switching  to  the  fee  b2  (M2  increases)  and  wait 
longer  before  switching  back  to  the  fee  b^  (the  difference 
(M2~m2)  increases).  This  shows  that,  to  the  contrary  of  the 
single  critical  number  policies  (see  end  of  Section  3),  the 
optimal  solution  depends  on  the  values  assigned  to  cost  and  fees. 

Observe  that  if  y=0,  m2  is  not  necessarily  equal  to  (M2-l) 
(figures  4 and  6). 


* 
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8.  Comparisons  Between  the  Two  Types  of  Policies 

To  conclude,  we  mention  numerical  comparisons  between  the 
two  types  of  policies  when  the  cost  y is  positive:  we  can 
determine  as  in  Sections  1 to  3 optimal  single  critical  number 
pol i ci es  . 

Figures  7 to  10  present  4 examples  corresponding  to  2 values 
for  N (N=5  and  N=10)  and  2 values  for  y [y=.5  - which  is  small 
compared  to  b1  and  bg  - and  y=5  - which  is  large  compared  to  b^ 
and  b£).  On  each  figure,  curves  1 and'  2 correspond  to  y=*5, 
curves  3 and  4 to  y=5;  curves  1 and  3 correspond  to  control  with 
hysteresis,  curves  2 and  4 to  control  with  a single  critical 
number . 

Obviously,  controls  with  a single  critical  number  yields  less 
good  results,  since  they  are  special  cases  of  controls  with 
hysteresis.  As  can  be  seen  in  the  figures,  the  difference  can 
be  substantial.  Also,  it  appears  that  the  optimal  control  with 
hysteresis  is  efficient,  except  under  severe  ci rcumstances , 
such  as  small  value  for  N,  large  cost  y and  quite  large  r 
(or  small  e). 
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Appendi x 

Let  us  denote  by  rm  M the  expected  total  reward  during  a 
busy  cycle  given  m and  M.  Consider  a random  walk  on  {0,1,. ...L} 
with  absorbing  boundaries  at  0 and  L.  Transitions  are  from  n 
to  n+1  (step  to  the  right)  and  n to  n-1  (step  to  the  left)  if 
n?*0  and  L.  The  interval  of  time  between  two  steps  to  the  right 
(to  the  left)  is  negative  exponential  with  parameter  x(p). 

Because  of  the  memoryless  property  of  the  negative  exponential 
distribution,  the  interval  of  time  between  any  step  and  a step 
to  the  right  (left)  is  negative  exponential  with  parameter  x(u). 
Moreover,  P[n-m+l  ] = X/  ( x+u ) and  P[n->-n-l  ] = y / ( X+y ) (n^O  and  L).  Let 

an(L,X,y)=P[absorption  in  0( initial  state  is  n] 

g (L,X,p)=E[number  of  steps  to  the  right  before  absorption  in  0 
or  L | initial  state  is  n]. 

It  is  well  known,  see  for  instance  [2]  p.  314  that 

an(L,X,y)=(l-pL)-1(l-pL‘n). 

where  p=X/y. 

We  show  in  [3]  that 

gn(L,X,u)=(l-p)"1p[n-L(l-an(L,X,p))]. 

If  L tends  to  »,  we  get,  if  p<l, 

an(»,X,u)=l 

9n(».X  ,y)  = np(l-p)  ^ . 

To  determine  rm  M,  we  decompose  a busy  cycle  as  follows. 
First,  there  is  an  idle  period,  at  the  end  of  which  a customer 
enters  the  queue  and  pays  b-j . Then  a random  walk  begins,  with 
absorbing  boundary  at  0 and  M,  parameter  x1  , to  each  step  to  the 
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right  is  associated  a reward  b-j . If  absorption  is  at  0,  the 
busy  cycle  terminates.  Otherwise,  a cost  y is  incurred  and  a 
new  random  walk  begins,  with  absorbing  boundary  at  m,  parameter 
Xg,  reward  b2  for  each  step  to  the  right.  Upon  absorption  at  m, 
a cost  y is  incurred  and  a new  random  walk  of  the  first  type  is 
initiated,  etc...  Thus,  if  rm(rM)  is  the  total  expected  reward 
during  the  remainder  of  the  busy  cycle  given  that  the  queue  is 
in  state  m(M)  and  if  gQ  n(L,A,u)  (resp.  gL  n(L,A,y))  is  the 
expected  number  of  steps  to  the  right  before  absorption  given 
that  the  initial  state  is  n and  absorption  is  in  0 (resp.  L)  we 
get: 

rm , M-bl +b 1 a 1 ( ^ **1  *u)g0  i ( M , A i ,y)  + (l-a1(M,A1  , y ) ) ( b -|  g^  -j  (M,A-|  juJ  + r^-y) 

rM=b29M-m("’X2’w)+rm"Y’ 

rm=bl am^ ,X1  ,lJ^90,m^M’Xl  »y)  + ^1"am(M,xi  ,p)^(bl^M,m^M,Xl  ^ + rM"Y^ ' 

As  gn  = an9o,n+^1"an^gL,n  ’ one  gets  eventua1ly 


m 


^ (M-m)pl|,(p1-p2)*^  p^’1  (1-p-|  )[2y(l-p2)  + (M-m)  (b1P1-b2P2)] 


(1-P2)(1 


p1'p2H 

M-m , I 
-p-|  ) J 


(1-P2)(l-P?'m) 


1 


( A . 1 ) 


Let  M be  the  expected  length  of  a busy  cycle.  We 


determine  <om  ^ in  [3]  by  the  same  technique  we  used  to  determine 


to  u is  the  idle 
m ,M 


rm  M'  We  Pr°P°se  here  another  argument: 
period  (mean  1 / A ^ ) plus  the  amount  of  work  brought  by  the 
customers  during  the  busy  cycle.  To  get  this  amount  of  work,  one 
just  replaces  in  (A.l)  y by  0,  b-j  and  b2  by  1/y. 

This  leads  to 
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figure  4 
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figure  5 
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figure  6 
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